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Abstract 

Instruction librarians want clear data showing the effectiveness of our workshops as a way 
of demonstrating our value in education. This article uses instructional design approaches to 
show how to make specific changes when writing and measuring our learning outcomes to 
capture what we are doing in our sessions. Unlike instructors with classes that develop over 
several months, we are faced with unique challenges when conducting one-shot instruction 
sessions. By focusing our attention on student satisfaction and learning, we see ways to 
improve those sessions for everyone involved. In this essay, we provide examples and 
discuss how to write effective learning outcomes to answer specific questions about learner 
satisfaction and what the participants learned. In addition, we suggest ways to reform the 
evaluation and assessment questions that we use to reinforce our lessons. These methods 
can be used in both online and face-to-face environments. 
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Don't Use a Hammer When You Need a Screwdriver: How to 
Use the Right Tools to Create Assessment That Matters 

Introduction 

Imagine: You have been asked to teach a one-shot library workshop in a credit course. 

(N.B.: For the purposes of this article, we will refer to time spent with learners as 
“workshops”; they could involve a stand-alone session, multi-part sessions, or one part of an 
ongoing credit class.) You are unlikely to have access to student work after the workshop is 
over, but you would like to know how well the workshop went and what students learned. 
What would you do? Many of us use some kind of assessment tool and compare the answers 
against our stated learning outcomes for the workshop. 

There are numerous tools to determine if we have been successful and if students have 
learned something useful. The list includes surveys; pre- and post-tests; the “one-minute 
paper” (Cross, 1998; Grassian, 2001); student reflections; worksheets; “fist-to-five” or “give 
me five” (Fisher, 2007); short questions and answers; analysis of student bibliographies; and 
more. These forms of assessment are popular, they are quick, and we use them because we 
hope they prove that students are better prepared to complete an assignment after seeing us 
than they were before our session. While all of these tools are used with the noblest of 
intentions, we want to be clear about what they can realistically tell us about what actually 
happened during our instruction session. We believe that meaningful assessment of student 
learning, the kind that shows changes in information-seeking behavior, is nearly impossible 
to do in a one-shot workshop. We cannot teach every skill a student needs in one session, 
nor expect to quantify that the session changed how students do their research. Instead, we 
should focus our efforts on evaluating what is possible in a single workshop. 

This article provides a framework to help you assess one-shot instruction sessions. First, we 
explain the ABCD Model for writing learning outcomes (LOs). If each of your LOs 
incorporates most of these elements, you will find that they help to clarify the content of 
your instruction sessions and the questions that you will want to ask students at the end of 
your session. 
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Once you have developed clear outcomes that incorporate the elements described in the 
ABCD Model, you should consider how best to determine whether students have met them. 
We suggest employing the four-part Kirkpatrick Model to guide these assessment decisions. 
Although the Kirkpatrick model was developed and widely used in private industry for 
decades, it is applicable to library instruction because it helps clarify what can be evaluated 
and assessed given the time and resources at your disposal. 

It is impossible for instructors to change students’ research behavior after one interaction 
with them. You will learn more about what students learned, and what they thought about 
the class, if you use the instructional design models presented here. It is important to have 
reasonable expectations of what you can evaluate or assess, and then to focus on developing 
methods that provide that data. We hope this article will help you create assessment that 
matters by choosing the right tools to provide specific feedback. 

ABCD Model 

As librarians, we know that our workshops should be guided by learning outcomes (LOs), 
which are statements written by an instructor that describe what learners should be able to 
do as a result of instruction. LOs can guide workshop design to help focus on specific 
behaviors and/or skills that you have determined are important for students to demonstrate 
as a result of instruction. By taking time to consider the LOs for a workshop, you will reflect 
on the most important behaviors and/or skills necessary to fulfill the instructional goal, and 
you will be less likely to overwhelm students with extraneous information. Many educators 
rely on Bloom’s Taxonomy to create LOs. Developed in 1956 (Bloom & Krathwohl, 1956) 
and revised in 2001 (Anderson, et al., 2001), Bloom’s Taxonomy is used "to categorize 
intellectual skills and behavior important to learning” (Coffey, n.d.). The revised Bloom’s 
Taxonomy provides six categories of outcomes: remember, understand, apply, analyze, 
collaborate and create. Each level has verbs associated with it that describe the cognitive 
process used by a learner. For example, the "remember” category might include verbs like 
recall, define, list, etc., while the "create” category includes verbs like design, develop, 
investigate, etc. The goal is to get learners to apply the cognitive processes in the highest 
category possible: i.e., analyze, collaborate, or create. However, a learner will likely need to 
"remember” facts and "understand concepts” before they can achieve the higher order 
learning. Many educators use the verbs suggested in Bloom's taxonomy when writing their 
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outcomes. An update by Anderson and Krathwohl (2001) is widely used and is an excellent 
resource for appropriate verbs to use in writing your own LOs. 

While including the observable behavior, associated verbs, and cognitive scaffolding 
suggested by Bloom’s Taxonomy are important aspects of writing a complete LOs, there is 
still more that you can do with them. You can write them to guide your evaluation and 
assessment decisions. Many instructional designers and educators have adopted the ABCD 
Model to make LOs measurable as well. In this model, each outcome names “the Audience of 
learners . . . specifies the Behavior or capability to be demonstrated and the Conditions 
under which the behavior or capability will be observed . . . [and] the Degree to which the 
new skill must be mastered” (Heinich, Molenda, & Russell, 1989, p. 36). Addressing each 
element in this model provides a map of how you intend to teach and assess each outcome. 
We describe each ABCD Model element in detail below. 

Audience 

This part of the model asks the question, “Who are the learners?” The answer can be as 
broad as “learners” or “students,” or it may describe the audience more specifically: for 
example, “undergraduate students” or “microbiology graduate students.” To make the 
outcome most useful, the audience description should accurately describe the majority of 
learners in the workshop. 

Behavior 

Next, you need to address the question, “What do learners need to demonstrate to show 
they’ve achieved the outcome?” In other words, what do you want your learners to be able 
to do when they leave your workshop? As noted, Bloom’s Taxonomy offers vocabulary that 
can be used to describe optimal student behavior as a result of achieving the LO, and can 
help you begin to assess student learning. The key here is that the behavior needs to be 
observable so that it can be measurable. You can’t assess what you can’t see. For example, 
instead of writing a LO that says students will understand a concept, ask yourself what the 
concept of understanding really means to you, and use that word instead. It could be the 
ability to use a particular feature in a database, or to describe how truncation works. In this 
case, you would replace the word “understand” with “use” or “describe.” If you would like to 
measure this outcome, consider collecting student work or request to see a sample of the 
end product, so that you can assess how well your students performed. 
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Condition 

The condition statement is likely the most unfamiliar part of the ABCD Model for most 
librarians. The question to answer here is “Under what conditions do learners need to 
perform the behavior?” Another way to think about this step is to consider it as the control 
statement. An example of a condition statement could be: “...given a list of article databases 
with descriptions...” The key factor is that the condition specifies that learners will have a 
list of article databases with descriptions. If the statement omitted the phrase “with 
descriptions,” then the behavior to “identify relevant databases” would imply that learners 
would need to independently assess the relevance of the databases. The current wording 
(“.. .given a list of article databases with descriptions...”) specifies that you will assess the 
behavior of students identifying a relevant database for their topics given a list of databases 
with descriptions. A thoughtful description of the condition will guide you in how to teach 
the behavior and how to assess it. 

Degree 

Finally, you should ask, “To what degree do learners need to perform the behavior?” This 
part of the model is most applicable when you are assessing learners formally or if you have 
access to a work product. A degree statement specifies which percentage of the learners 
should achieve the outcome in order for you to know that the material has been learned. 
Because most librarians do not have access to student work or other formal assessments 
after a workshop, many LOs are likely to omit the degree statement. However, if you are 
teaching a term-long course or see the same students multiple times during a term, you 
should consider a degree statement. 

Here is an example of an ABCD outcome: “Given a research question and a Wikipedia 
article about the topic of the question, learners will be able to generate a list of at least three 
keywords or phrases for each concept represented in the question.” The audience is learners, 
the behavior is generate a list, the condition is given a research question and a Wikipedia article, 
and degree is at least three keywords or phrases. To clarify, the condition in this outcome states 
what the learners are given in the workshop. For example, if the condition statement was 
more general (e.g., “Given a topic, learners will be able to....”) instead of enumerating the 
specific tools used (e.g., “Given a research question and a Wikipedia article about the topic 
of the question...”), you would also need to change the assessment method for this outcome. 
If learners just have the topic, you are implying that they can have any research question and 
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use any sources to generate their list of keywords. This approach may be fine; however, it 
will be more difficult to assess whether learners achieve this outcome because you will not 
know which sources they consulted. When your condition specifies the source, you are 
guiding and prompting them about where to look for keywords, and you are therefore 
better able to assess if learners have achieved the outcome. 

Kirkpatrick Model 

Next, we will discuss how you can use expanded ABCD outcomes with the Kirkpatrick 
Model to improve assessment. We will use the interaction of these models to argue that 
what you can reasonably measure depends on how extensive your interactions are with 
students. By using Kirkpatrick’s levels with the ABCD outcomes, you can verify that LOs 
correspond to the appropriate amount of subject matter to cover in the time you are 
allotted. The four levels of evaluation will help you examine where the LOs fall with the 
model. Once you know the Kirkpatrick Level for your outcome, we offer suggestions for the 
types of assessments you might use with outcomes at that level. 

The Kirkpatrick Model is widely used in many industries for assessment and evaluation of 
training. This model uses four levels of evaluation developed in 1954 by Donald L. 
Kirkpatrick as a result of his dissertation in philosophy (Kirkpatrick, 1954). The robustness 
of his framework has been demonstrated by its use in fields as wide-ranging as “business, 
government, military, and industry” (Watkins, et al., 1998, p. 90). While the model is not 
widely used in library instruction, it is beginning to be recognized as a useful tool for 
evaluation (Kaplowitz, 2014). We will explain here how to implement it in information 
literacy and instruction. Throughout the Model’s 55-year history, the descriptions of the 
four levels have been updated to reflect current evaluation concerns; however, the original 
four levels remain the same and are still relevant (Kirkpatrick Partners, n.d.). For this paper, 
we use the following definitions: Level 1 corresponds with satisfaction of the students; Level 
2 with what the students learned; Level 3 with what the students applied in practice; and 
Level 4 with whether the students are information literate as a result. The sections below 
describe each of the four levels and provide examples of both well-written and incomplete 
(i.e. non-example) LOs for each level. The image below illustrates Kirkpatrick’s four levels, 
the question for each level, and the types of instruction for which it is recommended 
(Turnbow & Roth, 2014). 
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Figure 1 - Kirkpatrick's Four Levels & Library Instruction 




Level 1: How satisfied are your learners? 

Level 1 concerns learner reaction. Level 1 assessment allows instructors to ask questions and 
receive meaningful responses about what the learner feels about a workshop’s format, 
instructor, classroom set-up, etc. Issues of library anxiety and students’ confidence in using 
library resources can also be evaluated here. Examples of common Level 1 assessment 
techniques include end-of-class evaluations soliciting feedback from our learners. 

Table 1 - Examples of Level 1 Assessment 


Level 1 Example 

B. Level 1 Non-Example 

After attending a library workshop, 90% of 
students will feel more confident than they did 
prior to attending the workshop because they 
know where to get help with their research. 

Students will get help from the librarian. 


Examining these two outcomes through the lens of the ABCD Model, it is clear why 
Example A is more useful. First, there is a specific condition statement: "after attending a 


[ Perspectives ] 


Turnbow & Zeidman-Karpinski 
Don't Use a Hammer... 















150 


Communications in Information Literacy | Vol. 10, No. 2,2016 


library workshop," which Example B does not include. Second, asking learners to rate their 
confidence (A) is a measurable behavior, as opposed to the vague behavior “students will get 
help"( B). Finally, Example A includes a degree statement, “90X> of students,” an established 
benchmark, which Example B does not include. Using Example A, it is clear that if less than 
90% of students feel more confident they know where to get help as a result of your 
workshop, then the LO is not achieved. (Remember, this is your benchmark, so make it 
whatever feels right to you. If a smaller percentage seems more attainable, then use the 
smaller number.) 

End-of-class evaluations or feedback forms are effective ways to measure success with 
Kirkpatrick Level 1 issues. Will Thalheimer (2016) is a well-regarded expert on an 
evaluation called “smile sheets,” which he describes as “a set of questions provided to 
learners after training - or a portion of training - asking for learner’s perceptions of the 
value of the training” (p. 5). You can use a paper or an online evaluation survey to quickly 
solicit Level 1 feedback, which focuses on learner satisfaction. A simple survey format 
consists of two or three clearly-labeled replies and space for comment. Examples of Level 1 
questions include: 

• The presentation was clear and understandable. 

• I enjoyed the library workshop. 

• The instructor was engaging. 

• The classroom set-up and technology facilitated my learning experience. 

• I am confident that I know where I can get help with my research. 

Appendix A includes an example a Level 1 assessment tool in a library workshop using these 
questions. 

Many educators and librarians are familiar with Likert Scales, which are questions asking 
the responder to use a rating for their answer, usually on of scale of l-to-5 or l-to-10. 
Thalheimer (2016) observes that, while popular, Likert Scales are problematic as assessment 
tools for several reasons. To begin, if you are using a five-point scale (with 1 being the 
lowest score and 5 being the highest), it is almost impossible to understand the difference 
between each person’s rationale for choosing between 1 and 2 or between 4 and 5. More 
significantly, the kinds of questions you should ask for Level 1 assessment have to do with 
student satisfaction, which can and should be posed directly. Finally, depending on which 
tool is used for these questions, the mechanics of the survey layout may make student 
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responses confusing. As of this writing, the popular and free Google Forms does not allow 
users to label each point on a Likert Scale. (Using a five-point Likert Scale on a Google 
Form led one colleague to wonder why half of his students seemed to understand a point 
that he had students practice in multiple ways, while the other half remained disturbingly 
fuzzy on it. He discovered that many of the students were confused about which choice was 
which on the Likert scale.) If you must use a Likert Scale, consider using software such as 
Qualtrics, which allows you to label each point. If not, consider using pull-down menus, or 
better yet, radio buttons, which allow you to set up replies with clear labels. In the next 
section, we will explain how to use questions and any possible answers to get more specific 
information about what your students have learned. 

Level 2: What have students learned? 

Kirkpatrick Level 2 assesses student learning. This level is concerned with the knowledge 
and skills learners have acquired as a result of a workshop. Many of your LOs are likely to 
fall in this level. The most widely used way to assess knowledge and skill level at the end of a 
one-shot workshop may be administering pre- and post-tests; however, we think that other 
options, detailed below, are more instructive. 

A well-written LO will guide you in designing formative or summative assessments. 
Kaplowitz describes formative assessment as learning activities included during your 
instruction that can be used as mini-assessments (2014). Summative assessments are usually 
administered after instruction and are meant to “measure the overall effectiveness of 
instruction once it has been completed” (Kaplowitz, 2014, p. 123). Examples C, D and E all 
demonstrate Level 2 LOs for one-shot library workshops. 

Table 2 - Examples of Level 2 Assessment 


C. Level 2 Example 

D. Level 2 Non-Example 

Given an overview of six search strategies that 
one can use to modify a search (i.e. Boolean; 
limits; abstract; database subject and keywords; 
bibliography/cited references; times cited 
references; and related records), students will 
be able to use at least three of them to modify a 
search in a database of their choice for their 

topic. 

Understand how to use Boolean terms, subject 
headings, MeSH terms and peer-reviewed 
journals for advanced searching. 
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If we examine these two outcomes through the lens of the ABCD Model, it is clear how C is 
the better of the two. In addition to the condition statement present in Example C, it 
describes observable behavior that students will need to demonstrate. We can observe 
students using search strategies; we cannot observe them understanding search strategies. 
Given the circumstances of the one-shot session (very limited time with the students and no 
access to student work after the session), we cannot reasonably expect to make this outcome 
measureable, but at least we can observe the behavior during the workshop (see sample in 
class activity below). Example C also specifies that students need only use at least three of 
the six strategies demonstrated. This statement clarifies the degree to which the students 
need to achieve the outcome. While both Example C and Example D lack degree statements, 
this type of LO can be assessed in different ways (described below). However, you could 
include a degree statement and know that achievement would be based on a sample of 
student work. Alternatively, Example E demonstrates how to rewrite the outcome to 
include a degree statement and different assessment. 

A sample in-class activity for formative assessment (as illustrated in Appendix B) might 
include a worksheet that asks students to search a database and use three of six 
demonstrated search techniques. Students would note which technique they used and 
whether they thought it was useful. These worksheets can be collected by the librarian or 
turned in to the instructor. If they are turned in to the instructor, you could ask for a 
sampling of them at the end of the term to assess the knowledge and skills the students 
gained as a result of the workshop. 

A method for assessing student learning summatively might be a search log assignment in 
which students name the strategy they used and why they used it. You could require them to 
use only a few of the strategies demonstrated, since not all of them will be applicable to the 
assignment. If the workshop is a one-shot session during a term-long class, ask students to 
turn in search logs to the course instructor and request permission to review them. The 
benefit to this second strategy is that your assessment allows students the time to practice 
using the new knowledge and skills. As such, it is more reflective of actual learning than an 
immediate assessment. 

There are times when you may not be able to provide an in-class activity as described above 
and illustrated in Appendix B. In these situations, you can still use smile sheets as a way to 
conduct Level 2 assessment. The key here is to make sure questions are focused on 
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performance, not just feedback (Thalheimer, 2016). Example E below is an example of how 
you could rewrite the LO in Example C to address a performance-based smile sheet. 

Table 3 - Example E. 


E. Level 2 Example 


After attending the library workshop, 80% of students will indicate that they are able to use the 
search strategies taught (i.e. Boolean; limits; abstract; database subject and keywords; 
bibliography/cited references; times cited references; and related records), with more practice or 
at an adequate or proficient level. 


An example of an assessment question for this LO might look like this: 

As a result of this workshop, are you able to use the search strategies (e.g. Boolean; 

limits; abstract; database subject and keywords; bibliography/cited references; times 
cited references; and related records) taught to search a database for your topic? 

A. I am NOT ABLE TO USE the search strategies taught in a database for 
my topic. 

B. I have general awareness of the search strategies taught, but I will need 
MORE GUIDANCE to use them to search a database for my topic. 

C. I am able to use the search strategies taught, but I will need MORE 
PRACTICE to use them well to search a database for my topic. 

D. I am able to use some or all of the search strategies ADEQUATELY to 
search a database for my topic. 

E. I am able to use some or all of the search strategies at a PROFICIENT 
level to search a database for my topic. 

The answer choices that learners select for this question are focused on their impression of 
their ability as a result of your workshop. Our LO is written to say that if 80% of learners 
indicate options C, D, or E, then our instruction has been successful. Appendix C includes 
our reasoning for this approach and offers additional guidance on how you might interpret 
results from this type of question. 

A well-written LO will guide you in designing formative and/or summative assessment. It 
will also help illuminate how you could teach the content students will need in order to 
demonstrate the knowledge and skills achieved. 
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Level 3: What do learners apply in practice? 

Kirkpatrick’s Level 3 focuses on learner behavior. What do learners apply in practice? It is 
one thing to learn a new skill, but what most instructors are concerned with is whether 
learners are changing their behavior as a result of instruction. For example, students may 
have learned and even practiced truncation in your workshop, but are they actually using it 
when it’s appropriate to do so in their searches outside of the workshop? 

In our experience, most instructors want to be, and think they are doing Level 3 assessment. 
We argue that it is difficult, if not impossible to do Level 3 assessment in a one-shot 
workshop. Changes in research behavior require time and practice. While a one-shot 
workshop might be well taught, we maintain that it is unlikely to be sufficient for changes 
of this kind. Example F provides a sample of a Level 3 LO. 

Table 4 - Example F. 


F. Level 3 Example 


Given an overview of six search strategies that one can use to modify a search (i.e. Boolean; 
limits; abstract; database subject and keywords; bibliography/cited references; times cited 
references; and related records), students will use appropriate strategies for their topic as 
determined by a librarian reviewing their search log. 


Example F slightly is different from a Level 2 outcome (C); although the behavior is the 
same, what is being assessed is changed. Instead of asking participants to fill out a handout 
or complete an exercise as a measure of success, an expert is reviewing a search log. It is 
significant to note that this outcome focuses on what students are actually doing on their 
own outside of class. We are measuring change in research behavior: Are students making 
the decision to use the preferred behavior (i.e. using truncation)? 

While Level 3 might not be attainable in a one-shot workshop, it should be considered if 
you are teaching a term-long course or have multiple contacts with learners throughout a 
term or program. Contact with students over time allows you to create assessment activities 
that can reinforce desired skills and measure behavior change. 
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Level 4: Are learners information literate as a result of instruction? 

Kirkpatrick Level 4 is about results and return on investment. In our experience, most 
instructors want to know if learners are more information literate as a result of instruction. 
If we had confirmation that this was the case, we would be able to argue that an instruction 
program and our teaching efforts are producing a solid return on investment of time, 
energy, and resources. 

While instructors want to confirm their effectiveness with this kind of assessment, we argue 
that it cannot be done at the end of a one-shot workshop. Both Level 3 and Level 4 
assessments are better suited for term-long or embedded courses, or for curriculum 
development. A comprehensive assessment might administer a post-graduation information 
literacy skills test, survey and/or focus group. While this may not be realistic for all 
institutions, there are projects that are striving for Level 4 assessment across our field. 

We think the depth and scope of the Carleton College “Information Literacy in Student 
Writing” project is a wonderful application of Level 4 assessment. This project uses a rubric 
to evaluate multiple sophomore writing samples selected at random for evidence of 
attribution, communication, and evaluation of sources (Jastram, et al., 2014). It is a process 
that allows the instruction librarians to consider students’ habits of mind in an authentic 
learning environment. Based on their findings, the program can shape the instruction 
topics, and it targets the classes that may benefit from additional information literacy 
instruction. 

Another example of Level 4 assessment is Project Information Literacy (PIL), done with the 
University of Washington's iSchool. PIL coordinates large-scale data samples of students at 
different campuses across the U.S., studying their information-seeking behavior (Head, 
n.d.). For example, PIL looked at recent graduates and their employers to compile findings 
that revealed some significant differences between what employers expected recent college 
graduates to know, and how those graduates actually conducted their research. The new 
employees were surprised to find that the pace, amount of ambiguity, and lack of direction 
in their workplaces was different than what they expected to find (Head, 2013). These 
findings have implications for what information literacy programs can teach students while 
they are still in school, and they are unlikely to have surfaced in any summative or formative 
evaluations conducted in a single workshop or a credit course. 
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Conclusion: Use the right tools for the job 

We have outlined how to combine a number of instructional design approaches and tools 
not typically used in library instruction; we recommend that you rethink how to structure, 
evaluate, and assess one-shot instruction, and then deploy these approaches accordingly. 

The ABCD Model and the Kirkpatrick Four-Level Evaluation Model provide frameworks 
for you to improve LOs. We have also provided examples of how to evaluate and/or assess 
the outcomes using various tools, including smile sheets and in-class worksheets. With this 
knowledge, we urge you to stop trying to do too much in an instructional setting. Librarians 
know that assessment is important, but we need to be doing the level of assessment that 
aligns LOs and class format. When teaching a one-shot workshop, you should not expect to 
conduct Level 3 assessment effectively. Instead, focus on creating in-class activities and 
performance-based smile sheets that will result in the learning measurable with a Level 1 or 
Level 2 assessment. You can use student work samples from a workshop (even if it is after 
the term) to gauge progress toward Level 3. It is also unrealistic to assess every student; just 
focus on getting a sample. We are confident that the sample will be an excellent way to 
show your value and to improve your instruction. Remember that learning is a process and 
takes time. It is unlikely that you will achieve Level 3 assessment unless you are deeply 
embedded in a department, helping with curriculum/course design, or teaching a term-long 
course. If Level 4 assessment is what you think (or have been told) is the most important 
part of your instruction program, then design a study that can accomplish that goal. The fact 
that it cannot be done in a one-shot instruction session does not mean you failed as an 
instructor. While you are waiting to conduct those big studies, we encourage you to 
concentrate efforts on retrieving the data to support and improve instruction. If you follow 
the models presented in this article, you will be poised to write effective LOs aligned with 
realistic evaluations, and you will be prepared to assess them using the right tools. 
Ultimately, this practice will allow you to collect meaningful data that will improve practice 
and demonstrate value. 
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Appendix A - Example of Level 1 Assessment (End of Workshop Survey) 

These are good questions to use at the end of a workshop to address material covered and 
impressions, but not how students put their new skills to work. If your goal is to measure 
implementation, you should wait a few weeks before sending out your evaluations (Thalheimer, 2016, 
p. 46-49). 

I enjoyed the library workshop. 

□ I loved it 

□ I liked it 

□ I disliked it 

Comments (especially if your answer is dislike it): 


The instructor(s) was engaging. 

□ I was engaged for most of the workshop. 

□ I was engaged some of the time. 

□ I was almost never engaged for the workshop. 
Comments (especially if your answer is almost never): 


The classroom set-up and technology facilitated my learning experience. 

□ Yes 

□ No 

Comments (especially if your answer is No): 


I am confident that I know at least one way to get help with my research. 

□ Yes 

□ No 

Comments (especially if your answer is No): 


Please provide any comments to help us improve this class in the future: 

If you have any remaining questions or want a librarian to contact you for help, provide 
contact information here too. Anonymous questions will be sent to the class. 
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Appendix B - Sample Level 2 Assessment (In-Class Activity) 

Spend a few minutes reading the descriptions for the databases recommended on the 
Library web guide. Select at least two databases that you could use to search your topic and 
write about why this one is good for your topic. 

1. Database #1: 

Why you selected it: 

2. Database #2: 

Why you selected it: 

Search one of the databases you selected above using at least three of the techniques 
presented in the workshop: Boolean operators, limits, abstract, database subjects/keywords, 
cited references or related records. Describe what you did and whether or not you thought it 
was useful. 

1. Technique #1: 

Description: 

2. Technique #2: 

Description: 

3. Technique #3: 

Description: 

Identify at least one article from your searches that you would like to read. Use [Institution 
SFX tool] to determine if it is available through the Library. Write the citation in MLA 
format below and select how you would get the item from the Library. 

Citation in MLA format: 

Is this item available online, in-print, both or via Interlibrary Loan? 
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Appendix C: Sample Level 2 End of Workshop Assessment 

The example below illustrates how you could assess a learning outcome using an in-class activity and 
an assessment question. 

In our limited experience of using questions like these for end-of-workshop evaluations, it 
has become increasingly clear that the examples and practice in the workshop should 
illustrate the complexities or gray areas for a concept; otherwise, students will be over¬ 
confident in their ability to do the task. For example, if the learning outcome is for the 
learner to be able to identify when to use Boolean operators, it isn't enough to just ask them 
to define “Boolean.” Instead, ask what they expect would happen if they used Boolean for a 
particular search. How would the results differ than if they didn't? Make sure the questions 
are difficult enough to offer some hesitation or discussion. If they are too easy, students may 
be over-confident in their ability since all they've experienced are "successes" through 
instruction. Below is an example of a question that you could ask during a workshop. 

Question: Which of the search strategies below would yield the results in red? 



A) (concussion AND balance) OR sports 

B) concussion AND (balance OR sports) 

C) (concussion OR balance) AND sports) 

D) concussion OR (balance AND sports) 


After students have had time to practice the desired behavior (in this case, using Boolean), 
then you could use a Level 2 assessment such as the one below: 

Question: As a result of the library workshop, are you able to use Boolean operators (AND, 
OR, NOT) as a part of your search strategy? 
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A. I’m NOT AT ALL ABLE to use Boolean operators as a part of my search strategy. 

B. I have general awareness of how to use Boolean operators, but I will need MORE 
GUIDANCE to use them as a part of my search strategy. 

C. I have general awareness of how to use Boolean operators as a part of my search 
strategy for this assignment, but I will need MORE PRACTICE to use it in future 
assignments. 

D. I am able to use Boolean operators SOME of the time as a part of my search strategy. 

E. I am ALWAYS able to use Boolean operators as a part of my search strategy when it 
is appropriate. 


The table below provides guidance on how you could interpret the results of students’ answers to the 
question above. Column A provides the answer options, and Column B is a suggestion of how you 
could interpret these results. 


Column A: Answer Choice 

Column B: Proposed Standard 

A. I’m NOT AT ALL ABLE to use Boolean 

operators as a part of my search strategy. 

Unacceptable. If students selected this answer 
choice after one-shot instruction, it would be 

evidence that the instruction was not 

successful. 

B. I have general awareness of how to use 
Boolean operators, but I will need MORE 
GUIDANCE to use them as a part of my search 
strategy. 

Unacceptable. If students selected this answer 
choice after this one-shot instruction, it would 

be evidence that the instruction was not 

successful. 

C. I have general awareness of how to use 
Boolean operators as a part of my search 
strategy for this assignment, but I will need 
MORE PRACTICE to use it in future 

assignments. 

Acceptable. If students selected this answer 
choice after one-shot instruction, it would be 
acceptable (and expected). 

D. I am able to use Boolean operators SOME 
of the time as a part of my search strategy. 

Superior. If students selected this after one- 
shot instruction, it would be great, although 
unlikely. 

E. I am ALWAYS able to use Boolean 

operators as a part of my search strategy when 
it is appropriate. 

Unlikely. It is not likely that students will be 
proficient as a result of one-shot instruction 
unless they had prior search experience. 


Boolean question from Dr. Andy Karduna, Department of Human Physiology, University of Oregon. 
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